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ABSTRACT 

Measures of the implementation of an individualized 
curriculum in 52 classrooms at the second-grade level are presented. 
The study also uses the measures of classroca processes to help 
explain student achievement on standardizert tests of reading and 
mathematics. The classroom pr,ocess data ar£ further aaalyzed to see 
if systematic differences occur in implemeatation vhich are 
attributable to the ethnic composition of the sctiool or to the degree 
of urbanization of the school setting. (Author) 
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I ntroduction 

Evaluative research in education is a relatively new field of 
investigation (Rossi, 1972). Systematic research in the field has concen- 
trated in two areas. The first area consists of research related to product 
development, generally called formative evaluation. Unfortunately little of 
the applied research in this area finds its way into the public arena 
(Light, 1972), The second area consists of the massive long term research 
related to end products, designated summative evaluation (Scriven, 1972). 
In order to integrate these two functions, there is a need in education to 
document the implementation of school programs in a way which provides 
information to program devalopers about the effects of variation in class- 
room processes on outcomes, in the formative sense, while providing the 
same information to educational consumers in the summative sense. Stated 
slightly differently, informction about classroom process should be considered 
as a set of dependent variables to provide information to developers and as 
an independent variable which explains achievement to pro^'ide information 
for educational consumers. It is the purpose of this paper to consider 
information about classroom processes in a specific program of adaptive 
education in this way. 

Related Research . In part because of the newness of educational 
evaluation and in part because of the nature of the field, educational 
evaluation must be eclectic in its selection of theoretical basis. For 
example, this study draws its theoretical framework from three streams of 
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thinking. The first and niost significant contribution has corie from the 
literature on program evaluation. All of the mjor modelfi of program 
evaluation include some aspect of process as a significant component of 
the evaluation information required (Cooley, 1971; Stake, 19t7; Lindvall 
& Cox, 1970), The second area of influence comes from the line of research 
which related to classroom observation and the relationship between class- 
room variables and achievement. The major contributions of this field have 
been toward indentifying and in some cases solving the methodological prob- 
lems in obtaining stable, interpretable and useful measures of classroom 
process (Rosenshine, 1971; Stuff lebeam, 1971; Wang, 1973)* The third area, 
the specific context in which this study was done, is the research and 
development work on adaptive educational programs which is on-going at the 
Learning Research and Development Center (LRDC), The study is especially 
dependent on the thinking and definitions of adaptive education of Glaser 
(1968, 1971, 1973), and Resnick, Wang, and Rosner (1974), 

Purpose. The purpose of this study was to provide measures of the 
implementation of the LRDC's program of adaptive education in 52 classrooms 
located in 21 schools, and to relate specific variations in the implementa- 
tion to variations in educational achievement. Within that rather broad 
statement of intent, four major objectives were identified. First, to 
provide classroom process data which describe the implementation of the 
major dimensions of the educational program. Second, to contrast specific 
dimensions of the model as they are implemented in rural versu& urban 
settings and in predominately black, white, or integrated school districts. 
Third, to use these data to explain achievement test results on standardized 
measurement instruments. Fourth, to compare the implementation of the 
programs over a two-year period and contrast the explanative power of the 
process variables from one year to the next. 
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Model > The model of evaluation which is used in this study has 
been proposed by Cooley and Lohnes In a forthcoming book-^. Two evalua- 
tive questions which are asked by their approach are: \^at effects in 
achievement can be attributed to specific elements in an Innovative educa- 
tional program? And what effects in achievement are confounded with in- 
teractions between program elements and entering abilities? They assume 
that it is inappropriate to ignore differences in entering behaviors 
either by failing to attend to them at all or by totally removing both 
entering behavior and all correlated process variation from consideration 
through a partialling procedure. The Cooley-Lohnes Model is a synthesis 
of identified evaluative concerns such as: measuring the dimensions along 
which educational treatments vary, exsimining the confounding and unique 
contributions of input and treatment, decidinp upon the appropriate unit of 
analysis, selecting valued outcome measures, and dealing with correlations 
within the input, process and outcome data. 

There has been an assumption in most discussions of program evalua- 
tion that in summative evaluations it is acceptable to attribute differences 
in programs (Tatsuoka, 19 72) rather than to document consistency and varia- 
tion within and between programs. The problem with this assumption is 
that there is a need to examine both within and between differences in 
programs as they appear in the classroom, not just to assume the relative 
similarity of programs under the same name (Lohnes, 1973). The limitation 
of the type of evaluative model I am employing is that it is correlational 
in nature and it, therefore, relies on replication as opposed to control by 



ICooley & Lohutis in preparation. 

ERIC 



randomization for validation of findings. To some extent, however, the 
limitation is also an advantage. The Cooley-Lohnes approach views educa- 
tiona.^ ^^valuation as an on-going process. The specific evaluative effort 
re-^^orted in this paper has been going on for three years, although the focus 
of inforination reported covers only one academic year. The distinct advan- 
tage of the model is that it fits the realities of most educational settings 
(e.g., no control of assignment of treatment, etc.). Other models tend to 
sidestep the issue that classrooms are not corn fields, and that although 
many universities have so-called laboratory schools, they bear no resem- 
blance to experimental agricultural plots found in departments of agricul- 
tural research. 

Setting 

The study reported in this paper was conducted during the 1972-7 3 
academic year. It is a direct follow-up and expansion of a 1971-72 s*"udy 
(Leinhardt, 1974). The evaluative effort has been focused at the second 
grade level at all sites rather than at all grade levels at a few sites. 
The assumption is that there is greater consistency in terms of process 
within sites across grades than within grades across sites. 

Background . Tha Learning Research and Development Center ^las 
been one of 22 national Follow Through sponsors. Each sponsn in the 
system is responsible for implementing its program at chose sites which 
have chosen that particular model. The LRDC implements its reading and 
mathematics programs in seven Follow Through sites and two developmental 
sites. This means tiiat the prograr^ are implemented to some degree In 52 
second grade classrooms with an average of 5.7 classrooms per site. The 
LRDC Follow Through sites extend from North Dakota to Arkansas to West 



Virginia, Out of the total of nine sites, three are predominately urban, 
five are predominately rural and one is suburban. Four of the sites are 
predominately white, two are predominately black and three are racially 
integrated. No site has been included in the evaluatioi study unless the 
program has been implemented at that site for at least one year prior to 
this study. Therefore, the study which was conducted during uhe 1971-72 
year was based on four Follow Through s'.tes and two developmental sites 
with a combined total of 30 classrooms, while the present study was based 
on seven Follow Through sites and two developmental sites. 

Instruction al Set ting . The basic characteristics of the LRDC 
instructional model are that: (1) it provides an environment which is 
adaptive to the educational needs of the student, (2) i.he curricula are 
organized and presented in a way which attempts to teach and reinforce 
basic cognitive skills, and (3) the student directs and controls his or 
her own learning within the context of the curriculum. Behaviorally , this 
means that the specific subject matter areas are broken down into units and 
objectives which are hierarchically sequenced in a curriculum. Placement, 
curriculum embedded tests (CETs), and posttest are constructed for each unit 
so that students can be initially placed and their success in learning the 
material can be monitored. By using the placement, CETs, and posttests, 
prescriptions are written for each student on a daily, weekly, or monthly 
basis. If a student does not pass a GET, (s)he continues in the same area 
with additional practice work. If a student does not pass a posttest, (s)he 
is recycled through the appropriate sections of the curriculum and retested. 

The classroom normally has one teacher and one assistant or aide 
for 25 children. Usually, the teacher and assistant both circulate (travel) 
around the room during the work period, sometimes one will circulate while 
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the other administers or corrects tests or tutors small groups. While 
circulating, the teacher or assistant corrects the work being done, occa- 
sionally alters prescriptions, offers brief tutorial assistance, and 
supports the student emotionally and academically. In general the morning 
is divided into a work period in which assignments are prescribed and an 
exploratory period during which the children investigate curriculum related 
material on their own. Classrooms vary in the degree to which students control 
the decision points in their daily learning situation, such as: which sub- 
ject to stud}^ and when, which assignment to do first, when to work on formal 
curriculum, when not to, when to take a posttest, when to change units, etc. 
Thus, while the curriculum is relatively consistent from classroom to class- 
room, there are quite a few areas in which there can be differences in 
implementing the program. 

The differences which occur between classes that are implementing 
the LRDC's model of adaptive education can be classified into six major 
domains. They are: Context, or background information which is not con- 
trolled by implementation; Allocation of Time, or the distribution of time 
by subject matter; Assignment Procedures, or the degree to which assignments 
are tailored to individual needs; Student Progress, or the frequency and 
manner of monitoring student learning; Student Autonomy, or the type of 
decision-making opportunities students have; and Attendance. These six 
domains represent areas in which any classroom implementing this educational 
program would be expected to show variation. For example, time is used 
differently in many different types of classroom settings, some classes have 
open subject matter scheduling others assign blocks of time for each 
subject area. 
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Procedure 

During the 1972-73 school year, data were collected to help in 
the evaluation of the LRDC instructional model. These data were collected 
at seven Follow Through sites and two developmental schools all of which 
were using the LRDC curricula in math and reading. The data came from four 
sources t (1) aptitude and achievement data were obtained from three 
sets of standardized tests: the Lorge-Hagen-Thorndike Test of Cognitive 
Abilities (1968) administered in the fall, the Wide Range Achievement Test 
(liJRAT) (1965) arithmetic subtest, and the Metropolitan Achievement Test (MAT) 
(1971) reading subtests both administered in the spring; (2) a questionnaire 
on classroom process which was designed to tap the six domains listed 
above; (3) individual student curricular placement and progress records; and 
(4) the teacher's class schedule. The last three sources yielded information 
on 29 measures of classroom process. Figure 1 is a list of the 29 
variables and the domains which were tap|^d by them. 

Instruments . The majority of the variables listed (22 of 29) were 
obtained from the 19 item questionnaire, which was administered three times 
during the year. The questionnaire was answered by the classroom teacher 
in November, the local education specialist in February, and the LRDC 
consultants in April. Three different individuals filled out the question- 
naire to avoid over burdening any one of them. The data from the three 
administrations were then averaged. 

Four other variables were obtained from the student profile sheet 
which were cumulative and provided accurate and easily accessible informa- 
tion about student progress through the math and reading curricula. These 
sheets were kept by the classroom teacher or aide and were sent to the 
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Center four times during the school year. The remining three variables, 
concerning time allocations, were obtained from the class schedules which 
were sent to the Center at the end of the year. 

The questionnaire focused on gathering information frun six major 
domains. The questionnaire will be described briefly by domain. 

Classroom Cout ext. This domain consists of five variables: the 
teacher's total years of teaching experience, the teacher's years of experi- 
ence with this program, teacher's years of experience in working with the 
particular aide, the number of children enrolled in the class, and the number 
of usable square feet per child in the classroom, excluding furniture. 
While none of these variables relate directly to the LRDC's program, they 
certainly are likely to affect the results of that program. The number of 
years of experience which the teacher has had with the program indicates 
not only the facility with which a teacher can be expected to use the 
program, but also an indication of how happy a teacher is with it. The 
number of children enrolled ^nd the amount of space available -?er child 
are both variables which relate to the conditions under which learning 
will occur and they can be expected to affect both climate and specific 
elements of the program, such as the frequency and nature of feedback. 

Allocation of Time . The data gathered from 1971-72 indicated that 
there was a postive and significant relationship between the amount of titne 
spent in inathematics per day and the level of matheiriatics achievement 
(Leinhardt, 1974). During the 1972-73 year more extensive information was 
gathered on the way in which individual teachers and sites spent the time 
available to them. The effort to identify actual t:^.me allocation focused 
on two aspects. First, what percentage of the day is spent in specific 
modes of instruction such as: individualized instruction, siTiall group 
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Figure 1 
Classroom Process Variables 

Classroom Conte xt 

Total teacher experience in number of years 

Total teacher experience with the LRDC instructional model 
in number of years 

Teacher-aide team experience with the LRDC instructional 
model in number of years 

Number of children enrolled in the class 

Number of square feet ner child 

Allocation of Time 

Percentage of time SDent in individualized activity 
Percentage of time spent in small group activity 
Percentage of time spent ir large group activity 
The number of minutes spent in math 
The number of minutes spent in reading 
The number of minutes spent watching educational TV 
Blocked or open scheduling of subjects 
Use of math maintenance programs. 

Assignment Procedures in Mathematics 

Percentage of unique assignments — the percentage of the 
total assignments given which were different from any other 
given 

Percentage of blocked assignments — where a blocked assignment 
was three or more consecutive pages or boxes with less than an 
equal number of isolated pages or boxes 

Assignments are or are not given during exploratory time 
Assignments are or are not changed during the day 

ERIC 



Figure 1 (continued) 

Monitoring Student Progress 

Percentage of pretests used for assignments in mathematics 
Percentage of CETs used for assignments in mathematics 
Percentage of posttests used for assignments in mathematics 
The number of days since the last test of any type was given 
Initial student placement in the math curriculum 
Initial student placement in the reading curriculum 
Student progress in the math curriculum 
Student progress in the reading curriculum 

Provisions for Student Self-Direction 

Student control of access to exploratory 

Sum of check list of student self-direction for total class 



Attendance 

Number of days oi teacher absence 

Percentage of children present on sample days 
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instruction and large group instruction. Second, what were the approximate 
amounts of time per day during which a child was exposed to specific sub- 
jects such as math, reading, or educational TV, The last two variables in 
this domain relate to specific classroom practices with time allocations. 
The first deals with whether or not different students are working on dif- 
ferent subject areas during the same time period. The second deals with 
whether o lot the classrooms are using math maintenance programs. 

Assignment Procedures . This domain deals with an especially 
important aspect of individual t "^.ad education, the assignment procedures. 
It is through the accurate marching of student needs and curriculum content 
that much of curricular individualization is achieved. The first two 
variables in this domain are vhi' percentage of unique assignments (the 
percentage of assignments given which are different from any others given 
with respect to level and skill) and percentage of blocked assignments (three 
or more consecutive tasks or pages with less than an equal number of isolated 
tasks). These variables attempt to monitor the way in which the teacher 
makes the assignments in the class using the mathematics assignments as the 
sample of assignment practices in general. The next two variables are 
questions about assignment policies: Are exploratory assignments given, and 
are assignments changed during the day? This last variable is an attempt to 
measure, indirectly, how responsive (on a daily basis) teachers are to students. 
The assumption is that if teachers do occasionally modify assignments during' 
the day, they do so in response to specific student needs. 

Monitoring Student Progress . This domain deals with the classroom 
practices used by the teacher to monitor the movement of students as they 
work in and complete the mathematics curriculum. The LRDC instructional 
model provides for frequent systematic checks on the students' learning 
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progress. These checks are generally in the form of individualized 
self-administered tests. These tests fall into three main categories; 
pretests, curriculum embedded tests CETs) and posttests. It is of inter- 
est to know which kind of test the teacher is using for prescription pur- 
poses. It is also of interest to know how frequently the teacher is 
checking the child's progress. Developers expect that teachers will uso 
CETs about 60 percent of the time, pretests about 25-30 percent: of the tine 
and posttests about 10-15 percent of the time for prescribing^, or assigning 
purposes. It is also expected that children will take at least on^ test of 
some type at least once every 5 to 7 days. The next four variables in this 
domain deal with the placement (where the stuvlent started work in the curri- 
culum) and progress (the amount of work the student completed in the curricu- 
lum) of students in the math and reading curriculum. 

Pro visions for S t udent Autonomy . One of the continual long-range 
goals of the LRDC model is to develop independent learners. In accordance 
with this there are many ways in which a teacher can provide opportunities 
within the classroom for the student to learn to exercise autonomy. The 
most obvious opportunity for this to occur is during the exploratory time. 
But, there are other subtle ways during the prescription time in which the 
teacher can convey to the student their responsibility for their own learn- 
ing. The autonomy check list (Question 11 on the questionnaire) is an 
attempt to identify most of the opportunities for student autonomy which 
would exist in a classroom and list them. It is also the first step in 
developing a specific measure of student autonomy. 

Teacher and Student Attendance . Both tearuer and student atten- 
dance are important variables relating to the in^plementation of the LRDC 
program. If the teacher is consistently absent, the chance cf the program 
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being correctly implemented, and of children making steady educational 
progress goes down« If the students are not present in school, they cannot 
generally benefit from instruction. Although an individualized program 
makes student absenteeisn less damaging than in traditional ones, it is 
still a matter of concern when attendance is low. Good attendance can also 
be taken as an indication of a positive attitude toward school in general. 

The assignment of variables to the domains which have been dis- 
cussed is to some extent arbitrary. For example, when the general domain 
of allocation of time wis considered the specific question concerning 
whether or not different subjects were taught at the same time or not was 
raised, and so the variable is considered in this discussion to belong to 
that domain. However, that particular variable could be considered to 
belong to other domains as well, such as: management proceedings, assign- 
ment procedures, or even student autonomy. The point is that the variables 
and domains are not to be interpreted as a rigid set of definitions, but 
as a convenient way of considering a rather large number of classroom 
process variables. Thus, it is expected that the domains may be expanded 
or contracted as more information about classroom practices is gained or 
as the utilization of the information changes. 

Reliability 

When an investigation moves from the pure confines of measurement 
theory and classical experimental design into the relative disorder of the 
classroom, a host of questions concerning the validity of the findings arise 
While classical test construction settings can assume that the underlying 
characteristic being measured remains stable and only the precision of 
measuring it varies, no such assumption can be made for any measurement of 
classroom characteristics. 
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\\fhen classroom characteristics are measured, not only must the 
instrument's reliability be estimated, but also the stability of the 
characteristics itself should be investigated. This problem and its solu- 
tion have been noted by anyone who is seriously concerned in investigating 
classroom processes, such as Rosenshine (1971), Ross (1974), and Keeves (1973) 
to mention only three, l^iat has also been noticed is the extreme difficulty 
in obtaining high estimates for classroom stability, which frequently makes 
results ambiguous. 

For the purposes of this study, it appears userful to report three 
estimates of reliability: inter-coder reliability, instrument reliability, 
and the classroom stability. Inter-coder reliability (Pearson product 
moment correlation) was estimated to be ,98. Instrument reliability 
(Pearson product moment) was estimated to be ,90. Instrum.ent reliability 
was considered to be the reliability of variables which should not change 
over the year such as: number of years of teacher experience (teaching, in 
the program, and with the aide), the size of the room, and the space allo- 
cated for exploratory. The correlation was calculated between the fall and 
spring questionnaires. The classroom stability using Cronbach's (1963) 
intra-class correlation was ,76, This was calculated on the variables which 
could he expected to change during the year. 

Both instrument reliability and classroom stability are confounded 
with intern-respondent reliability because different individuals filled out 
the questionnaires. This was done intentionally, not only to distribute the 
burden of responding to the questionnaire, but also to give usefvjl feedback 
to the Follow Tlirough staff itself about the difterences between the teachers*, 
specialists \ and LRDC consultants* knowledge and perceptions 
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of the classrooms involved. As a primary purpose of collecting the Oata was 
to gain information not estimate reliabilities of measures, this seemed like 
a legitimate procedure. 

Findings 

This section addresses four questions. First, in a descriptive 
sense, what does a program of adaptive education look like when it is 
implemented in a network of schools as contrasted to what developers 
thought it should look like? Second, in a comparative sense, in what way 
are differences in implementation systematic with respect to differences 
in the location and composition of those sites? Third, how successful is 
a generalized construct of process variables in explaining differences in 
student achievement which are not explained by differences in student input. 
Fourth, how similar are the results over a two-year period? 

Program Implementation 

One purpose of this study was to provide information about the 
implementation of the major dimensions of the educational program and to 
compare the implementation to previous years. This information must be 
provided before any more sophisticated analysis of the data can be inter- 
preted. The clearest and most direct way of presenting these data are by 
means and standard deviation by site. \^/hile the raw data by classroom is, 
of course, available, it v;as felt that it would be more comprehensible and 
interpretable to examine nine school sites on 29 variables than to examine 
54 classrooms on 29 variables. However,, in order to preserve a sense of 
the variations within sites, the standard deviations presented represent the 
standard deviations of the classroom means rather than the means of the 
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Tables 1 through 6 present the results by site. Each table pre- 
sents the data for one of the six domains listed in Figure 1. The data for 
each table will be briefly discussed. 

Con text V ar lab 1 e s . In Table 1 the site averages on five context 
variables are presented. It can be seen from the first variable, the number 
of years of teacher experience, that the sites vary tremendously. Site 1 
teachers have an average of 2.2 years experience while Site 4 teachers have 
an average 22,4 years experience. The next two variables, the years of tea- 
cher experience with the program and the number of years which the teacher/aide 
team has been working together, are highly dependent on the number of years 
that LRDC has been involved with individual districts. That is, it is nearly 
impossible for a teacher to have more years experience with our program than 
the number of years we have associated with that site. In spite of that, one 
can see from Table 1 that Sites 3-5 with which we have been involved the 
same number of years have differences in the amount of experience their 
teachers have had with our program. Sites 1, 3, 4, 8, and 9 which were 
examined last year have increased the number of years of teacher experience 
with the program. This indicates in an indirect way teacher satisfaction 
with the program. 

The fourth variable, the number of children enrolled, appears to 
spilt into two groups, each of which is consistent within sites. One group 
has an average size of about 25 children and the other has about 20 children 
which is consistent with the class sizes in 1971-72. The fifth variable, 
the number of square feet, seems to vary quite a bit both within and across 
sites » Isfhile classroom size does not generally affect achievement directly 
except at the extremes, it certainly does affect the use of exploratory space 
and time and the type of interactions which a teacher will engage in with 
students. 



Allocation of Time . The first three variables on Table 2 deal with 
the percentage of time spent (estimated) in each mode of instruction: indi- 
vidual, small group, large group. Naturally, we would expect to find that 
approximately half of the day is spent in individualized activity, because 
the majority of LRDC's instructional program is individualized. Also, we 
would expect the least amount of time to be spent in large group activity. 
From Table 2 it is clear that there is noticeable difference between the 
sites. Most sites spend between 2/3 and 3/4 of the day in an individualized 
instructional mode. However, sites 1 and 3 spend slightly less than half of 
their time in individualized instruction. 

The second assumption is that the least amount of time should be 
spent in large group activity. This assumption is based on the fact that 
the early part of the reading program specifies a good deal of small group 
work. However, the data show that five of the sites soend more time in 
l?.rge group activity than in small p.roup. Of course, it should be remembered 
that this information refers to the entire school day, not just the three and 
one-half hours during which the program is implemented. In sites 2, 5, 
and 7 where more than half of the day is spent in individualized activity 
it means that the sites are individualizing more than just those subjects for 
which LRDC provides curriculum material (sites 8 and 9, however, do have 
additional Center material). This is an encouraging and exciting sign of 
commitment on the part of teachers and schools to individualization. 

The amount of time spent on math and reading deaily were related to 
achievement in 1971-72. It can be seen from Table 2 that there is quite a 
bit of variability across districts on chese two variables. The amount of 
time spent on mathematics stayed approximately the same for all sites 
observed last year. The amount of time spent on reading went down slightly 
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for three (i, 3, and 4) of the six sites observed last year, while site 8 
went up considerably. The amount of nime spent watching educational TV was 
recorded because we were interested in knowing which, if any, of the sites 
nad chosen to use educational television as a regular part of the academic 
day. It appears that those classrooms which do v/atch educational TV, do so 
very infrequently and as a result, little affect on achievement is expected. 

The next variable, different subjects studied during the same time 
period, appears to occur in some classrooms at most sites. (The exceptions 
are sites 5 and 9 where all classrooms report block scheduling.) This is 
interesting because the Center is beginning to systematically investigate 
the different effects and relative advantages of open and blocked scheduling 
(Wang, 1974). The last variable, whether or not a math maintenance program 
is used, indicates that the majority of classrooms in most districts use 
some form of math maintenance. The utility of a uniform math drill program 
is a question of considerable concern among implementors . It is hoped that 
these data might shed some light on its utility. 

Assignment Procedures . The first two variables reported in Table 3, 
the percentage of unique assignments and the percentage of blocked assign-- 
ments, indicate the way in which the teacher makes the assignments in the 
class. It appears that the teachers in the sites are making very unique 
assignments. In fact, the assignments overall are 30 percentage points more 
imique than that reported in previous years. The percentage of assignments 
which are blocked is, however, also quite high for all districts ranging 
from 47 percent to 89 percent. In the mathematics curriculum (from which 
both of these variables are taken) the guidelines suggest that only those 
pages or boxes which are needed as a result of failure to pass specific items 
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on a test be assigned. However, it may be that the amount of time required 
to prescribe so specifically is not worth the additional effort once the 
initial starting point in a curriculum is identified for a student. 

The next two variables, exploratory assignments given and assign- 
ments changed during the day, both occur in the majority of classrooms in 
the majority of sites. Recently, there has been an effort to identify the 
activities in the exploratory area more closely with curriculum. It appears 
from the data that the exploratory time and materials are now more tightly 
related to the curriculum. It should be noted, however, that the item in 
the questionnaire asked if any assignments are given — so the positive reply 
does not indicate that all exploratory activity is now controlled by the 
teacher. The last variable is an attempt- to measure how responsive (on a 
daily basis) teachers are to students. The assumption is that if teachers 
do occasionally modify assignments during the day, they do so in response to 
specific student needs. It is, therefore encouraging to see the majority of 
teachers stating that they do this. 

Monitorin g Student Pr o gress . The data on the monitoring of student 
progress are presented in Table 4. It can be seen from all examination of 
the first three variables that the sites are very close to developer expecta- 
tions in Che teacher *s relative use of types of tests. For all sites, 
except site 8, the GET is the most frequently administered test (71.8% to 
29.4%); the pretest is the next most frequently used test (49.8% to 15%); 
and the posttest is the least frequently used test (18.5% to 6.33%). CETs 
are the most frequently administered test because they are the most frequent 
type of test in the curriculum. The number of days since the last test also 
falls within the expectations of the developers with a range of 3.4 days 
to 9. 7 days. 
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The next four variables in Table 4 deal with the placement and 
progress of students in the math and reading curriculum. Placement refers 
to the unit in which the child started working in the fall; progress refers 
to the number of units mastered during the year. It is evident from the 
table that there is a large amount of variation between districts in the 
placement of students in math and reading (4,2 to 20,9 and 3,0 to 14,4, 
respectively). This is a reflection of the basic difference in entering 
ability among the students in different sites. It is also supportive evidence 
of the need for adaptive education. In addition, there is difference between 
districts on the amount of student progress in the math and reading curricula; 
however, it is not nearly so extreme as the difference observed in initial 
placement (7,9 to 16,8 units progress in math and 5,0 to 8,1 books progress 
in reading) , 

Provisions for Student Autonomy , The first variable in Table 5 
deals with the question of access to exploratory, while the second variable 
represents the sum of the number of items checked off on the autonomy check 
list (a maximum possible score is 14), Table 5 indicates that at most sites 
a minority of teachers gave students the control of going to exploratory. 
The second variable indicates that the potential opportunities for autonomy 
the teacher utilizes relatively few. There seems to be a fairly strong 
site effect on this variable with marked differences among sites and smaller 
differences within sites. 

Teacher and Student Af:tendance, In Table 6 the average teacher 
absences (a day is the unit of measure) as obtained from school records are 
reported. Table 6 also reports student attendance based on the attendance 
on three randomly selected days. Teacher absences vary between sites and 
within sites quite extensively. The extremely high mean and standard 



deviation at site 5, 12,0 and 24.0 respectively, is largely due to a single 
teacher who was seriously ill for most of the spring semester. The data on 
student attendance are for the most part encouragingly high. Three of the 
sites remain at about the sane level of attendance as last year, site I's 
attendance is seven percentage points higher than that reported last year. 
However, site 8 is 12.0 percentage points lower than last year. This infor- 
mation has been passed on to site 8 as a matter of some concern. 

Summary, The function of the previous discussion was to document 
and describe some of the more important aspects of the implementation of 
LRDC's educational program. Three major points have been made. First, that 
while classrooms differ considerably in the basic contextual facilities 
available to them, and to some extent in their use and allocation of the 
program materials available to them, they are reasonably consistent in the 
implementation of key aspects of the program. Second, that although the 
LRDC program is quite structured and defined in terms of teacher /student 
roles and the use of the curriculum there is also a considerable amount of 
leeway for classrooms to adapt the program to fit their particular needs. 
Third, it is hoped that this information has served to be somewhat 
descriptive of the basic elements of the program. 

Contrasts of the Setting 

A second goal of this paper was to examine specific dimensions of 
the model as they are implemented in rural versus urban settings and in 
settings which have different racial compositions. The reason for contrasting 
the implementation of the program in diverse settings is evaluative* It 
would appear significant, for example, if the program of adaptive education 
was "exportable*' to certain types of schools only, either predominately 
white or black schools, or to certain areas, such as cities rather than 



rural settings. In fact, it seems mandatory for any argument supporting 
an innovative educational program to show that it is not limited in that 
way, but rather that it can be implemented successfully in a variety of 
settings. It is also important, however, to be able to demonstrate that 
the program has enough flexibility to adapt to the needs and values of 
different educational and community situations. 

In order to test whether or not the program has been implemented 
equivalently in the various school sites, two one-way multiple analysis 
of variances were performed using the NYMBUL mult i variance computer program 
Version 5 (Finn, 1972), The program was run on 51 of the 52 classrooms. 
One was eliminated because of the inability to appropriately assign it to 
any cell. The two one-way analyses were run rather than a two-way because 
of two empty cells — black-rural and integrated-urban. 

BefOiTe discussing the results, it is useful to delineate which 
variables were chosen for which reasons and how the sites were grouped for 
each contrast. For the first contrast, urbanization, the classrooms were 
divided into two groups: rural and small town population centers of less 
than 21,000; and urban populations ranging from 75,500 to 520,000, In one 
boarder-line case of assignment, the nature of the surrounding economy 
(farming) was taken into consideration (e,g,, farming versus industry). 
There were 32 rural clasfsrooms and 19 urban classrooms. For the second 
contrast the classrooms were divided into three groups: 11 predominately 
black classrooms, 17 predominately white classrooms, and 23 integrated 
classrooms (included in the integrated classrooms were 6 predominately 
Indian classrooms). In order to deal with the limited degrees of freedom 
available within cells, ten variables were selected from the intital pool 
of 29 to be used for this analysis. The variables were chosen from each 



domain excep;. context. The context variables were omitted because, in 
general^ they cannot be controlled although they are variables which undoubt- 
edly affect the implementation of ones which can be controlled. Table 7 
lists the ten variables which were used in this analysis by domain. Table 7 
also indicates the exact probability levels for the univariate F's for the 
contrasts In question. (Note; Tha stepped-down F*s in no case contradicted 
the univariate F's at the .05 level.) 

The results reported in Table 7 indicate that the program is being 
implemented at a consistent level independent of makeup or location of the 
schools involved. For seven of the ten variables, there is no significant 
(a=.05) difference for either comparison. l\/hile tb.is means that the program 
for the most part is being implemented consistently in a variety of educa- 
tional settings, it does not mean that there is no difference between 
individual districts and classrooms. It does mean that those differences 
are not systematic by geographical or racial setting. The variables which 
are consistent across settings are: Percent of individualized time, the use 
of a math maintenance program, the percent of unique assignments, the number 
of days between tests, student progress in mathematics, opportunities for 
study autonomy and the number of days of teacher absences. These results are 
especially exciting for two of the va.iables: math progress and teacher 
absence. Equivalent student progress in the presence of vastly different 
entering abilities is a very impressive statement in support of the effective- 
ness and efficiency of individualized education. The second variable of 
special interest, teacher absence, could be considered a morale indicator. In 
general, teachers in urban settings tend to have higher absenteeism and turn- 
over than those in rural settings (Havighurst, 1968). Teachers using this 
program seem to be contradicting this trend. 
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Three of the variables, however, showed differences for these con- 
trasts. In general, urban, black and integrated schools spend more time in 
mathematics, give more pretests and fewer CETs. Looking more closely at the 
means for the time spent in mathematics: Urban schools average ") minutes 
a day in mathematics while rural schools spend 67 and 62 minutes in mathe- 
matics while white schools spend 53 minutes in it. (Although it was not 
included in the analysis, a similar trend was found for time spent in reading.) 
The decision of how much time to allot for each subject is largely a site 
decision, which is often modified by individual teacher actions. The LRDC 
guidelines suggest that at least 60 minutes a day be spent in math so that 
these data are indicative of some failure to implement this aspect, with 
rural and white schools getting roughly three hours less math time a month 
than the other schools. 

The next two variables, the percentage of pretests and CETs which are 
used to make assignments need to be examined together, because the results 
tend to balance each other out. Urban schools use more pretests than rural 
ones, but give fewer CETs. It is appropriate to consider the situation in 
this way because the use of posttests for prescription indicates that the 
instructional sequence has been ineffective for a child for Lhat particular 
unit. For example, urban schools used 34 percent pretests and 4 7 percent CETs 
for prescription, while rural schools use 21 percent pretests and 67 percent 
CETs for prescritpion. Or stated another way, approximately 81 percent and 
88 percent of the prescriptions are continuation rather than recycling prescrip- 
tions. (Of course, the CETs can result in additional work in the same area, 
but generally they do not result in complete recycling.) 

Summary . The purpose of the previous discussion was to show that 
while there are differences between classrooms on many variables of imple- 
mentation, these differences are not systematic with respect to geographic 



location of school and racial makeup of the classrooms, and it is encouraging 
to find that this is in fact the case. Of course, only 10 of the 29 vari- 
ables were examined; however, the ten were chosen to be representative of 
the major domains of interest. There are other contrasts whcch could be made 
which also might be of interest, such as comparing experienced (2 or more 
years wich the program) to inexperienced sites to see if the program becomes 
substantially more integrated into the particular educational settings over 
time. 

Up to this point, the discussion has focused on the level of 
implementation of the LRDC program as a dependent set variables without con- 
sideration of the effect of the implementation on student achievement. 
Using the previous description of the educational settings as a background, 
we will now consider classroom processes as they relate to student achieve- 
ment on academic tests. That is, we will consider classroom process as 
the independent set of variables which effect achievement. 

Classroom Process and Student Achievement 

Several approaches can be used to examine the relationship between 
student achievement and classroom process. Two approaches will be examined 
and their relative strengths and weaknesses discussed. The first approach 
is to use some form of simple correlational analysis. The second approach 
is to use partial correlations wiiich control input. Either of those procedures 
would result in p. Iciige matrix (29 x 29) of correlations between each of the 
process variables, input, and outcome measures or of the correlations between 
the residuals of the process and outcome measures. There aiv:. two major 
problems with either correlational approach. The first is that there is too 
much data to interpret and absorb. The second is that frequently there is a 
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high correlation between input, process, and outcome which makes inter- 
pretation very ambiguous. For example, schools which have students with 
high entering abilities frequently have genuinely better school practices. 
If one fails to control the input, then it is not clear whether the classroom 
practices or student ability effects achievement. If, on the other hand, 
input is partialled out in this situation, genuine relationships between 
process and outcome are lost. In summary, there are two problems which 
need to be solved in an analysis of the data: The data must be reduced in 
a way which does not lose information (such as merely selecting one or tv;o 
variables from a domain) , and the procedure can neither ignore nor eliminate 
the effects of student input on process measures and achievement. 

As mentioned earlier, Cooley and Lohnes have been actively involved 
in finding a solution to this set of problems. They propose using a modifi- 
cation of John Carroll's model for the investigation of the educational 
process which is generalizable across educational settings and independent 
of che specific subject matter domain under investigation (Carroll, 1963), 
Of course, there are many possible ways in which the data can be grouped 
and examined, the selection of this particular one is largely a matter of 
convenience* Cooley and Lohnes also provide an analytic procedure which is 
an extension of Beaton/ s Commonality Analysis (Beaton, 1973) and which deals 
with the problem of accounting for input, process, and outcome information. 

Model. The Cooley-Lohnes model consists of six components. Each 
component and the specific set of variables from the data set, which were 
used to construct the components, are listed below: 

1, Initial input: "is a construct used to reflect the 

basic incoming skills of the children in the classroom 



(Cooley & Emrick, p. 5).'* This domain is repre- 
sented by the sum of the Lorge-Thomdike test and 
initial student placement in mathematics, averaged 
across students at tlie classroom level. 
Opportunity : is a construct which describes the 
time available to an individual student which is 
allocated to activities which are directly related 
to subject matter acquisition. This domain is 
represented by the sum of: time allocated per 
day to mathematics, time allocated per day to 
reading, the percentage of time spent in individualized 
activities, the smallness of the class size, and 
the use of exploratory assignments. 
Motivation : "is a construct used to reflect the 
pupil ^3 tendency to engage in learning activities 
when the opportunity exists/' (Cooley & Emrick, 
^97^\y p. 6). It is perceived as a construct which can 
reflect external (e.g., curricular and interpersonal) 
and internal influences. However, in this case the 
construct was not measured directly, and it is repre- 
sented by the sum of the autonomy checklist score 
and student attendance. 

C urricular Structure : is a construct designed to 
measure the degree to which a curriculum is organized 
and sequenced (linearly or not) and the degree to 
which a student and curriculum are successfully 
matched. This construct is represented by the sum 



of the following variables: the percentage 

of assi(?;nments which are unique, the percentage 

of pretests given, the percentage of CETs given, the 

number of days sines the last test, the nunJ^er of 

unit^ the student progressed in math, and whether or 

not math maintenance is being used, 

5. Instructional (teaching) Event : is a construct which 
is designed to measure the frequency, quality, and 
duration of instructional contacts. In the present 
data, this dimension v;as not adequately measured. The 
variables included in this construct are the sum of 
the following: the percentage of assignments which 
were not ones involving recycling of the students, the 
number of years of teacher experience and experience 
with the program (because experience has in the past 

been correlated with better teaching practices [Leinhardt, 
1974] ), the number of days in which the teacher was 
present . 

6, Outcom e: is the end of year performance of students 
on the reading section of the Metropolitan Achievement 
Test and the math portion of the Wide Range Achievement 
Test . 

This model provides a convenient, reduced structure by which the 
data can be grouped for ease in interpretation. The distinct advantage of 
the model is its relative simplicity and generalizability . It is clear 
that the model is being used post hoc in this case and that the variables 
are being "fitted" into it. However, th'.-^ advantages of the clarity of 
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interpretation are felt to outweigh tne disadvantages of a less than elegant 
fit between models and variables. 

In addition to providing a conceptual schema for considering the 
data, the model provides a statistical one for analyzing the data. Essen- 
tially, the procedure is one of multiple regression in which the total 
variance explained by the model is partitioned into variance uniquely 
explained by one or more elements and variance which is commonly explained 
by the group of predictors. Figure 2 represents a simple case of the 
statistical model being used. IVhile the figure is self-exDlanatory in 
the simple case, it should be obvious that in the case of multiple 
process variables, areas 2 and 3 on the figure become complex very 
rapidly. 

Results. Table 8 presents the intercorrelat ions among the six 
constructs. It is worth noting that the four process constructs are 
positively related to both input and outcome measures; this implies that 
"good" classroom practices are confounded with high entering abilities on 
the part of students. Table 9 presents the results of the multiple 
regression of the first five predictors (input plus process) on the 
sixth variables, outcome. The top row represents the total r2 for each 
of the five predictors. The second row represents the portion of total 
variance explained which is unique to each construct. The third row 
represents the variance explained confounded among all predictors in the 
system. The first row is the sum of the second and third rows. 

The results in Table 9 clearly indicate that the most powerful 
predictor of student achievement in the spring is student abilities in the 
fall. The relationship between fall and spring scores explains 65 percent 



Figure 2 

Schema of the Partitionine 
of Explained Variance into 
Unique and Common Elements 




l=\^ariance in achievement 
explained uniquely by 
entering performance 

2=Variance in achievement 
explained uniquely by 
educational process 
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3=Variance in achievement 
explained by both process 
and entering performance 
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of the variance in spring performance. However, of the variance exolained, 
39 percent is unique to the input construct, while 26 percent is confounded 
with the other predictors. 

The combined process variables explain 35 percent of total vari.^nce 
in achievement; however, an even smaller portion of that variance, 7 percent, 
is a result of the unique contribution of classroom processes. In examining 
Table 9 more closely, an interesting example of the utility of this type of 
analysis appears. The two process constructs, curricular structure and 
instructional event, appear to explain similar amounts of total variance, 
17 percent and 16 percent respectively. However, the construct of instruc- 
tional event is totally confounded with the other predictors (15.8 percent). 
The curriculum structure construct, however, is approximately two-thirds 
confounded and one-third unique. 

The results from this analysis indicate that while classroom process 
information does not explain a large amount of the variance in end of year 
achievement, it certainly explains a portion of it. The specific contribu- 
tion of each of the 22 variables is, of course, not considered here; however, 
it is hoped that the constructs have been identified clearly enough to permit 
rhe recder to understand the relative importance of cluster of variables to 
the explanation of student achievement, 

A Comparison with Previous Results 

Earlier in this paper it was mentioned that the evaluation of 
classroom processes is an ongoing concern. Therefore, data from 19 71-72 
are also available for comparative purposes. The data collected in 1971-72 
are not identical to those collected in 1972-73: however, they have many 
variables in comiTJon or they are similar. Figure 3 lists the data for both 
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Figure 3 

Comparisons by Construct of 1971-72 and 1972-73 Data 

1972 - 73 Data 1971 - 72 Data 

.V=52 N=30 



IMPUT 

Lorge Thorndike 
Math Placement 

OPPORTUNITY 

Time allocated for Math 
Time allocated for Reading 
Mumber enrolled 

Percentage of individualized time 
Exploratory assignments given 

MOTIVAilON 

Attendance 

Sum of autonomy check list 



STRUCTURE 

Percencage unique assignments 
Number of days since the last test 
Student Math progress 
Percentage of pretests 
Percentage of CETS 

Presence of a Math maintenance program 

INSTRUCTIONAL EVENT 

Teacher program experience 
Teacher total experience 
Number of days teacher was absent 
Percentage of post-tests 



INPUT 

Lorge Thorndike 



OPPORTUNITY 

Time allocated for Math 
Time allocated for Reading 
Number enrolled 



MOTIVATION 

Attendance 

Child gets own work 

Play follows work 

Percentage of negative statements 

STRUCTURE 

Percentage unique assignments 
Number of days since the last test 



INSTRUCTIONAL EVENT 

Teacher program experience 



Number of cognitive statements 



OUTCOMES OUTCOMES 

URAT Math WRAT Math 

MAT Rend 
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years grouped by construct. The figures shows that for the most part the 
data gathered in 1971-72 is a subset of the data gathered in 1972-73. The 
exception is that classroom observatJon in 1971-72 yielded data not avail- 
able in 1972-73; i.e., the percentage of negative statements, the number of 
cognitive statements, and the distribution of all teacher contacts over 
students . 

Table 10 shows the intercorrelation among the six constructs for 
data collected during 1971-72, In this case, two of the constructs, oppor- 
tunity and structure, are only slightly related to input, but both are related 
to outcomes . 

Table 11 gives the total , unique, and confounded variances explained 
by each construct for the 1971-72 data. The overall is .607, which is 
lower than the overall r2 observed in 1972-73. The total variance explained 
by input is 47 percent with 19 percent uniquely related to outcomes. This 
is probably due to the fact that the Lorge-Thomdike test is not as powerful 
a predictor as the Lorge-Thorndike plus initial student placement in the 
curriculum, which was used in 1972-73. The total variance explained by the 
process constructs is 41 percent; however, only 14 percent of that is unique. 

Three things should be pointed out from this analysis. First, in 
boEh years input explains achievement better than process, both in total 
variance explained and in unique variance explained. Second, it appears 
that is is easier to improve the explanative capacity of input measures than 
it is to improve measures of classroom process. Third, Tables 9 and 11 indi- 
cate a considerable amo\mt of fluctuation in the explanative capability of 
the specific process constructs. In the 1972-73 data, structure and instruc- 
tion appear to be the most useful variables, while in 1971-72 opportunity 
and motivation are most useful. This is partly due to the changes of 
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specific variables over the Uwo years (see Figure 3), but more important, 
it is due to the fact that this is a post hoc fit between variables and model. 
In order for the specific process constructs to be interpreted, measures 
with high construct validity will have to be developed; that is, measures 
specifically designed to tap each construct. Fourth, regardless of internal 
fluctuation within the model, classroom process variables appear to be 
relatively stable over time and setting in their ability to explain observed 
variance in student achievement. 

Summary and Conclusions 

In this study 1 have attempted to deTnonstrate several points. First, 
I have shown that a program of adaptive education can be implemented in a 
Variety of educational settings. Second, I have shown that the classroom 
processes which surround the implementation can be identified and measured 
and that these measures provide information which is descriptive of the 
program and of the setting; thus providing information to educational con- 
sumers about key aspects of the program while providing information to 
developers about both the context in which their products are used and on 
the transformations which their products undergo. Third, I have shown that 
the program of adaptive education whila varying in its implementation by 
classroom does not show systematic variation in success of implementation 
with respect to the ethnic composition of the school or the degree of 
urbanization of the school setting. 

Finally, I have shown that the measures of classroom process are 
useful in helping to explain variance in end of year student achievement. 
However, it still appears that it is far easier to improve the explanative 
power of measures of student input than it is to improve the explanative 
power of measures of classroom process. Of course, process is the area in 
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which we as educators have manipulative control. Therefore, regardless of 
whether or not we can substantially increase the amount of variance 
explained, it is still the major portion of the educational environment 
over which we have qualitative control and responsibility. The full impact 
of classroom process on student achievement will only be clearly measurable 
and interpretable when the relationship between student entering abilities 
and educational process is reduced. 

The results of this study raise some interesting issues for further 
research. The major issue concerns the utility of a generalized model of 
educational process. It appears to me that even though specific information 
about the nature of new programs can be of considerable value to both con- 
sumers and producers of educational products, there is a very great need to 
provide additional information which is independent of the specific programs 
involved. Such information should include specific measures of the teacher's 
and students* behaviors and curriculum usage with regard to: the opportunities 
for knowledge to be gained, the motivation for gaining that knowledge, the 
structure of the learning environment, and the quality of instructional 
events or interactions (Cooley and Lohnes, in preparation.). By designing 
measures of classroom process which are both universal enough to describe a 
variety of educational programs and specific enough to describe unique 
differences among situations, we can hope to improve our knowledge of which 
education practices affect which educational outcomes. 
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Table 8 

Tntercor re lacion Among Six Kducational Constructs 
for 19 72-7 3 Data \'=52 

Input Opportunity Motivation Structure Instruction Outcome 



Input 

Opportun i ty 
Mot i va t ion 
S true ture 
Ins true t ion 
Ou tcome 



L.OO 
.03 
.15 
.19 
.48 
.80 



I. 00 
-.25 

.30 
-.09 

.12 



1.00 
- . 22 
.01 
.12 



1.00 
.17 
.41 



1.00 



1.00 



Table 9 

Total, I'nique, and Confounded Variance Explained 
hv Construct R-=»72 :n = 52 

Input Opportunity Motivation Structure Instruction 



Tot.ll 
Var lance 
Exp la ined 

Vn ique 
Var iance 
Explained 

Variance 
Expla ined 
Con founded 
wi til 
Other 

Pred ic tors 



.646 



39J 



,0U 



,001 



.014 



,005 



170 



.06^ 



158 



.000 



,255 



.013 



.009 



. 107 



158 
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Table 10 

into r c o r V c .1 a t i o n A pio n g Six Kc ! u c a t ion a 1 on s t. r uc t s 
for 1971-72 Data N=30 

Input 0 p po r t un i t V ^iotivation Structure Instruction Outcome 



Input 1.00 

Opportunity .06 

Motivation .42 

Structure -.01 

Instruction .45 

Outcoiae .68 



1.00 
.26 
-.11 
-.06 
.33 



1 . 00 
10 
-.07 



1.00 
-.04 
.16 



1. no 

. 31 



1.00 



Table 11 

Total, Unique, and Confounded Variance Explained 
by Construct r2=.606 N=30 



Input Opportunity Motivation Structure Instruction 



Total 
Var lance 

Explained .466 .111 .170 .025 .097 

Unique 
Variance 

Explained .198 .082 .010 .04 5 .006 

Variance 
Explained 
Con founded 
with 
Other 

Predictors .268 .029 .160 -.020 .091 
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